1. Technical Field
The present technique relates to imaging apparatuses capable of reducing white band interference arising when the flash of a still camera or the like (an external flash) occurs when capturing a moving picture for a video.
2. Description of the Related Art
In recent years, imaging apparatuses that use CMOS (Complementary Metal Oxide Semiconductor) image sensors, which are compact, consume little power, capture images at high speeds, and so on, are being employed in the fields of consumer video cameras and professional video cameras.
A CMOS image sensor has various properties that are different from a CCD (Charge Coupled Device) image sensor, and the method for reading out charges accumulated in a photodiode (also denoted as “PD” hereinafter) differs as well.
In a CCD image sensor, PD charge readout is performed simultaneously for all of the pixels, which is what is known as a global shutter system. On the other hand, in a CMOS image sensor, the PD charge readout is carried out using what is known as a rolling shutter system, in which the readout timing shifts progressively line by line (pixel by pixel). For this reason, CMOS image sensors suffer from deficiencies not found in CCD image sensors because skew in the readout times of accumulated charges causes the timings of the accumulation periods for respective pixels to shift.
One such example of such a problem is a phenomenon in which white band interference occurs when a subject illuminated by the flash of a still camera or the like is captured using a video camera that employs a CMOS image sensor. “White band interference” refers here to a phenomenon in which only part of a frame in a captured image is affected by the flash, resulting in a bright image above or below a line partway through the image (that is, the upper portion of a screen or the lower portion of the screen).
This phenomenon will be described hereinafter using FIG. 9 and FIG. 10.
FIG. 9 is a diagram illustrating a scene in which both a video camera and still cameras are capturing a subject, such as a scene at a press conference.
FIG. 9 illustrates a scene including a video camera 10, a monitor 11 that displays the captured signal therefrom, still cameras 12 and 13, and a subject 14. Note that the video camera 10 employs a CMOS image sensor.
When the flashes of the still cameras 12 and 13 are used in this scene, white band interference appears in the screen of the monitor 11 that displays the captured signal from the video camera 10. The principles behind this will be described hereinafter.
FIG. 10 is a diagram schematically illustrating the charge accumulation periods (exposure periods), the readout timings, and the scanning periods of the video camera 10. FIG. 10 expresses the charge accumulation periods of the respective scanned lines that configure the screen and the scanning periods for reading out those charges using the horizontal axis as the time axis. Assuming an HD camera, the total number of scanned lines is 1,125.
Here, the frame rate for imaging is 24 frames per second.
“Monitor screen 0 interval” refers to the interval in which the captured signal of a frame 0 is output to the monitor screen or the like; here, this interval is 1/60 second. The same applies to a “monitor screen 1 interval” and so on.
For example, in a line 1, which is the uppermost line in the screen (that is, one line's worth of pixels in the imaging element of a CMOS image sensor for obtaining a video signal that forms the line 1 (with a PD provided for each pixel)), the PD accumulation (that is, the accumulation of a charge by the PD) of a frame 1 commences at exactly the time at which the monitor screen 0 interval starts, and thus the PD accumulation ends after one accumulation frame interval, or in other words, at the time at which the monitor screen 1 interval starts.
Immediately thereafter (that is, immediately after the PD accumulation has ended), the readout scanning of the accumulated charge in the accumulated PD signal of the line 1 is started, and at the same time, the PD accumulation of the following frame 2 commences (note that “accumulated charge readout” is sometimes referred to simply as “readout”). Because 1,125 lines are scanned in one output frame interval ( 1/60 second), the PD signal readout scanning period is 1/60/1,125≈14.8 microseconds.
Next, a line 2 commences PD accumulation at the time at which the PD readout scanning period of the line 1 in frame 0 ends. In other words, the line 2 carries out the PD accumulation and readout operations after a delay equivalent to the PD readout scanning period after those operations are performed for the line 1. The same operations as described thus far are carried out for line 3 and on.
Thus with the rolling shutter system, the charge accumulation periods of the lines of which a single frame is configured shift little by little from top to bottom, as illustrated in FIG. 10. In accordance therewith, the scanning periods of the respective lines, or in other words, the PD signal readout timing, occur immediately after the charge accumulation periods of those lines, as illustrated in FIG. 10. In other words, with the video camera 10 that employs a CMOS image sensor, the PD signal readout process is carried out sequentially in line order, with the PD signal of the line 2 being read out after the PD signal of the line 1 has been read out and so on.
Here, as shown in FIG. 10, when a flash occurs near the middle of the monitor screen 1 interval (the interval denoted as a “flash emission interval” in FIG. 10), the bright light from the flash affects the charge accumulation periods of the latter lines in frame 1 and the charge accumulation periods of the former lines in a frame 2. The flash light that occurred in the monitor screen 1 interval spans across the charge accumulation and charge readout timings of lines X and Y in frames 1 and 2, as shown in FIG. 10.
In other words, the affect of the bright light of the flash is as follows in the case illustrated in FIG. 10.
(Lines a1 of frame 1 (the lines belonging to the area indicated as “a1” in FIG. 10):
In frame 1, the area of the lines a1 before the line X is not affected by the flash light (that is, the charge accumulation period has already ended).
(Lines X to Y of frame 1 (the lines belonging to the area indicated as “a2” in FIG. 10):
The area of the lines a2 in the period from lines X to Y is affected by the flash light in frame 1, and the amount of accumulated light gradually increases.
(Line Y and on in frame 1 (the lines belonging to the area indicated as “a3” in FIG. 10):
The area of the lines a3 from the line Y on is affected by the total light amount of the flash light.
(Lines b1 of frame 2 (the lines belonging to the area indicated as “b1” in FIG. 10):
Conversely, in frame 2, the area of the lines b1 before the line X is affected by the total light amount of the flash light.
(Lines X to Y of frame 2 (the lines belonging to the area indicated as “b2” in FIG. 10):
The area of the lines b2 in the period from the lines X to Y is gradually affected less by the flash light.
(Line Y and on in frame 2 (the lines belonging to the area indicated as “b3” in FIG. 10):
In the area of the lines b3, from line Y on, the accumulation period has not yet started, and thus those lines are not affected by the flash light.
Therefore, when the period in which the flash light is emitted is only an instant and the transient periods in the areas a2 and b2 in FIG. 10 are small enough to be ignored, generally speaking, the lower half of the monitor screen 1 (the screen (image) formed by the captured signal from frame 1) is bright, as illustrated in the lower section of FIG. 10, whereas the upper half of the monitor screen 2 (the screen (image) formed by the captured signal from frame 2) is bright; this appears in video display apparatuses as white band interference. Unlike a CMOS image sensor, in an imaging apparatus that employs a CCD image sensor, the charge accumulation times of all of the lines of which a single frame is configured are the same, and thus this problem does not arise; instead, a natural image in which the entire image brightens when a flash is emitted appears.
There is thus a problem with imaging apparatuses with a CMOS image sensor in that white band interference occurs in the captured signal when an external flash of light such as a flash light or the like occurs.
The imaging apparatus disclosed in JP-2007-306225A (called “Patent Document 1” hereinafter) exists as a conventional imaging apparatus for solving this problem.
FIG. 11 is a block diagram illustrating an example of the configuration of a conventional imaging apparatus 9000. The imaging apparatus 9000 is a digital still camera that primarily records what are known as still images.
As shown in FIG. 11, the imaging apparatus 9000 includes an image capturing unit 701, an image processing unit 702, a recording display processing unit 705, a buffer 706, an evaluation unit 703, a storage unit 707, and a control unit 704.
With the conventional imaging apparatus 9000, for example, when a still image or a moving image has been captured by the image capturing unit 701 in response to a user operation, the captured image undergoes a predetermined image process in the image processing unit 702, and is then supplied to the recording display processing unit 705 and the evaluation unit 703.
The recording display processing unit 705 buffers, in the buffer 706, the image that has undergone the predetermined image process in the image processing unit 702, and the evaluation unit 703 generates an evaluation value for the image using a detection circuit and supplies that evaluation value to the control unit 704. The control unit 704 then temporarily stores the evaluation value in the storage unit 707.
A computation circuit in the control unit 704 calculates a difference value between that evaluation value and an evaluation value that is already stored in the storage unit 707, or in other words, the evaluation value generated from the image of the previous frame. If that difference value is greater than or equal to a pre-set reference value, the image is determined to have been negatively affected by an external flash, whereas if the difference value is less than the reference value, the image is determined to not have been affected by an external flash. Based on the result of the determination, the control unit 704 controls the various elements of the imaging apparatus 9000 so that, in the case where the image has been determined to have been negatively affected by the external flash, that image is discarded, whereas in the case where the image has been determined not to have been negatively affected by the external flash, that image is output.
In this manner, the conventional imaging apparatus 9000 solves the problem of white band interference caused by an external flash.
Furthermore, in order to solve the aforementioned problems, another method used by an imaging apparatus, for example, adds together two frame images that have been affected by external flash and replaces frame images that have been affected by the external flash with a frame image generated by adding the two frame images together, thereby eliminating images having white bands occurring due to the external flash.
However, with the aforementioned conventional technique, the influence of the external flash is corrected through the generation of a new image by adding together the two images that have been affected by a flash, and thus in the case where an electronic shutter function, which is a function of an imaging apparatus, is employed, there are situations where the influence of an external flash cannot be properly corrected.
FIG. 12 is a diagram schematically illustrating the charge accumulation periods (exposure periods), the readout timings, and the scanning periods of a video camera in the case where an electronic shutter function of the video camera is used. With this video camera, the charge accumulation period is 1/24 second in the case where the electronic shutter function is not used; however, FIG. 12 illustrates a case where the electronic shutter function is used and the charge accumulation period is 1/48 second. In other words, in the video camera illustrated in FIG. 12, the accumulated charges are discarded in exactly half of the charge accumulation periods during normal imaging when the electronic shutter is not used. In the video camera illustrated in FIG. 12, the charges accumulated in 1/48-second periods, from the PD signal readout time of a frame N (where N is an integer) to the PD accumulation start time of a frame N+1 (that is, the periods indicated by the dotted line quadrangles in FIG. 12), are discarded.
Here, as shown in FIG. 12, when a flash occurs near the middle of the monitor screen 1 interval (the interval denoted as a “flash emission interval” in FIG. 12), the bright light from the flash affects the charge accumulation periods of the latter lines in frame 1. Accordingly, as in FIG. 10, the lower half of the monitor screen 1 (the screen formed by the captured signal from frame 1) is bright, as illustrated in the lower section of FIG. 12; this appears in the monitor as white band interference.
However, in the case of FIG. 12, white band interference caused by the influence of the flash does not occur in the upper half of the screen (the monitor screen 2). The reason is that in the video camera, due to the electronic shutter operation, the charges accumulated in the 1/48-second periods from the PD signal readout time of frame 1 to the PD accumulation start time of frame 2 are discarded, and thus are not used as image signals.
Accordingly, in the case of FIG. 12, the influence of the flash does not appear as white band interference in the image of frame 2 (the monitor screen 2 (the screen formed by the captured signal in frame 2)). In other words, in the case of FIG. 12, the video (image) captured by the video camera includes a screen in which the lower half is bright white, but not a screen in which the upper half is bright.
For this reason, in the case of FIG. 12, even if processing according to the stated conventional technique is carried out, two screens (frame images) that have been affected by a flash cannot be added together, and thus the influence of flash light cannot be properly corrected.
In order to solve the aforementioned problem, it is an object of the present technique to provide an imaging apparatus, an external flash correction method, a program, and an integrated circuit capable of obtaining an image (video) in which the influence of a flash is properly suppressed even in the case where the image has been captured using an electronic shutter function.